Skip to content

🏃 e2e: Rewrite kubernetes main branch conformance to use clusterctl #1760

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 13, 2020
Merged

🏃 e2e: Rewrite kubernetes main branch conformance to use clusterctl #1760

merged 1 commit into from
Jul 13, 2020

Conversation

@randomvariable
Copy link
Member

@randomvariable randomvariable commented Jun 23, 2020 •
edited
Loading

Signed-off-by: Naadir Jeewa [email protected]

What this PR does / why we need it:
Moves the conformance test to be part of the new clusterctl e2e test suite, which should make it easier to maintain in the long term.

test/e2e_new/conformance is designed to be copied to Cluster API core. Some concurrent testing has been done with capd.

Additional changes to e2e suite include:

  • Cleanup clusters when hitting ctrl+c
  • Continuously dump namespaces on the bootstrap cluster every 5s
  • k/k is not imported so separate go mod files are not required
  • SSM session manager plugin downloaded by make, and consumed to scrape commands from EC2 instances without an SSH key.
  • Accepts kubetest viper configurations as input
  • Conformance can run in parallel, bringing test time to sub 45 minutes

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1610
Fixes #1587
Fixes kubernetes/kubernetes#91860

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 23, 2020
@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jun 23, 2020
@randomvariable
Copy link
Member Author

randomvariable commented Jun 23, 2020

/test pull-cluster-api-provider-aws-make-conformance

@randomvariable randomvariable force-pushed the conformance-refactor branch 2 times, most recently from b6b0070 to f2e1a0b Compare June 23, 2020 12:20
@randomvariable
Copy link
Member Author

randomvariable commented Jun 23, 2020

/test pull-cluster-api-provider-aws-make-conformance

@randomvariable randomvariable mentioned this pull request Jun 23, 2020
Closed
@randomvariable randomvariable force-pushed the conformance-refactor branch 3 times, most recently from 577aeca to 991d5c1 Compare June 24, 2020 16:13
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 24, 2020
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 24, 2020
@randomvariable
Copy link
Member Author

randomvariable commented Jun 24, 2020

/test pull-cluster-api-provider-aws-make-conformance

@randomvariable
Copy link
Member Author

randomvariable commented Jun 24, 2020

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 24, 2020
@randomvariable randomvariable changed the title [wip] 🏃 e2e: Rewrite kubernetes main branch conformance to use clusterctl 🏃 e2e: Rewrite kubernetes main branch conformance to use clusterctl Jun 24, 2020
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 24, 2020
@randomvariable
Copy link
Member Author

randomvariable commented Jun 24, 2020

Pending a passing run on conformance

randomvariable
randomvariable commented Jun 24, 2020
View reviewed changes
randomvariable
randomvariable commented Jun 24, 2020
View reviewed changes
randomvariable
randomvariable commented Jun 24, 2020
View reviewed changes
@randomvariable
Copy link
Member Author

randomvariable commented Jun 24, 2020

/test pull-cluster-api-provider-aws-make-conformance

@randomvariable randomvariable removed the request for review from sethp-nr July 8, 2020 12:28
@randomvariable
Copy link
Member Author

randomvariable commented Jul 8, 2020 •
edited
Loading

OK, conformance is passing, but I have a small Makefile change to do.

@randomvariable randomvariable force-pushed the conformance-refactor branch from 2c8058b to 6880b00 Compare July 8, 2020 15:16
@randomvariable
Copy link
Member Author

randomvariable commented Jul 8, 2020

Done

@randomvariable randomvariable force-pushed the conformance-refactor branch from 6880b00 to 3b1480c Compare July 8, 2020 17:53
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 8, 2020
@randomvariable randomvariable force-pushed the conformance-refactor branch from 3b1480c to a891ab7 Compare July 8, 2020 18:02
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 8, 2020
@randomvariable
Copy link
Member Author

randomvariable commented Jul 8, 2020

rebased

@randomvariable randomvariable force-pushed the conformance-refactor branch from a891ab7 to 6c3e04b Compare July 9, 2020 00:00
@randomvariable
Copy link
Member Author

randomvariable commented Jul 9, 2020

And again

@randomvariable
Copy link
Member Author

randomvariable commented Jul 9, 2020

/test pull-cluster-api-provider-aws-make-conformance

@randomvariable randomvariable force-pushed the conformance-refactor branch from 6c3e04b to 92aa94b Compare July 9, 2020 14:50
@randomvariable
Copy link
Member Author

randomvariable commented Jul 9, 2020

Rebased again. The test/e2e_new/kubetest is temporary, with the plan to move it into CAPI's test framework as part of kubernetes-sigs/cluster-api#3306

vincepri
vincepri reviewed Jul 9, 2020
View reviewed changes
detiber
detiber reviewed Jul 9, 2020
View reviewed changes
Copy link
Contributor

@detiber detiber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes look good to me, could you link to a completed conformance test using this change? I'd like to see what the prow output looks like

@randomvariable
Copy link
Member Author

randomvariable commented Jul 9, 2020

@detiber https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cluster-api-provider-aws/1760/pull-cluster-api-provider-aws-make-conformance/1276074049079873536 is a typical run.

I then renamed $(ARTIFACTS) to $(ARTIFACTS_DIR) in Makefile and didn't realise that was an injected variable, changed back now.

@randomvariable randomvariable force-pushed the conformance-refactor branch from 92aa94b to 21ab1c9 Compare July 9, 2020 15:29
@randomvariable
Copy link
Member Author

randomvariable commented Jul 10, 2020

rebased

@randomvariable
Copy link
Member Author

randomvariable commented Jul 13, 2020

ping @detiber @vincepri

@detiber
Copy link
Contributor

detiber commented Jul 13, 2020

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 13, 2020
vincepri
vincepri approved these changes Jul 13, 2020
View reviewed changes
Copy link
Member

@vincepri vincepri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Jul 13, 2020

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vincepri

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 13, 2020
@randomvariable
Copy link
Member Author

randomvariable commented Jul 13, 2020

/test pull-cluster-api-provider-aws-test

@k8s-ci-robot k8s-ci-robot merged commit a6070d1 into kubernetes-sigs:master Jul 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

3 more reviewers

@dims dims dims left review comments

@detiber detiber detiber left review comments

@vincepri vincepri vincepri approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@detiber detiber

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Milestone

v0.5.5

5 participants

@randomvariable @detiber @k8s-ci-robot @dims @vincepri